# DPO fine-tuning

Chocolatine-2-14B-Instruct-v2.0.3 is a large language model based on the Qwen-2.5-14B architecture, fine-tuned with DPO, specializing in French and English tasks, and excels in the French LLM leaderboard.

A fine-tuned version based on VAGOsolutions/SauerkrautLM-gemma-2-9b-it, optimized for Italian language performance with support for 8k context length.

The Sunfall model is a natural language processing model developed based on Llama-3.1-8B-Stheno-v3.4, suitable for specific functions and application scenarios.

Tanuki-8x8B is a large-scale language model pretrained from scratch, optimized for dialogue tasks through SFT and DPO

This is a Llama-3.1 model fine-tuned with DPO, which can behave more 'humanely' and avoid the rigid responses of AI assistants. It is suitable for role-playing scenarios.

Ko-Gemma-2-9B-IT is a Korean dialogue model in the Gemma model series, based on supervised fine-tuning (SFT) and direct preference optimization (DPO) training of google/gemma-2-9b-it, specifically optimized for Korean text generation tasks.

Jamet-8B-L3-MK.V-Blackroot is a role-playing and storytelling model based on Llama 3, optimized through multiple iterations and particularly suited for creative writing and interactive dialogues.

A DPO fine-tuned model based on Llama-3-8b, focused on enhancing truthfulness and mathematical reasoning capabilities

A large language model fine-tuned via DPO based on Meta-Llama-3-70B-Instruct, demonstrating excellent performance in multiple benchmark tests

Noro-Hermes-3x7B is a Mixture of Experts (MoE) model built using the lazy merge toolkit, combining three 7B-parameter Mistral variant models with capabilities in intelligent assistance, creative role-playing, and general task processing.

Korean vocabulary expanded version based on SOLAR-10.7B-v1.0, a large language model fine-tuned with DPO

The first Polish dialogue assistant model specifically designed for local deployment, based on TinyLlama-1.1B with extended Polish tokenizer and trained with DPO optimization

This is a small 7B-parameter LLM optimized for device-side use, pruned and trained with DPO

NeuralHermes is a large language model based on OpenHermes-2.5-Mistral-7B, further fine-tuned through Direct Preference Optimization (DPO), demonstrating excellent performance across multiple benchmarks.

Mini_synatra_7b_02 is a large language model fine-tuned by Minirecord based on the maywell/Synatra-7B-v0.3-dpo model, specializing in text generation tasks.